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An  Agent-based  Model  Simulation  of  Multiple  Collaborating  Mobile  Ad  Hoc 

Networks  (MANET) 

Abstract 

The  paper  presents  a  preliminary  result  of  using  cognitive-  and  behavior-based  modeling 
framework  to  simulate  a  network  of  MANETs  (Mobile  Adhoc  NETworks)  as  intelligent 
agents  in  a  tactical  battlefield.  We  demonstrate  the  efficacies  of  an  agent-based  modeling 
and  the  need  for  developing  formal  methods  for  multi-agent  simulations  from  a  system  of 
systems  (SoS)  perspective.  The  development  of  our  model  framework  informs  the  need 
to  describe  behaviors  and  relationships  of  actors  and  objects  in  the  context  of  a  mission 
space,  and  2)  to  provide  a  foundation  for  modeling  agent  behaviors  in  a  way  that  is 
plausible  with  respect  to  human  behavior,  a  specially  from  the  standpoint  of  human- 
system  interactions. 

1.0.  Introduction 

The  surge  in  the  human  dependency  on  Mobile  Ad  hoc  NETworks  (MANET)  in 
various  task  scenarios  (battlefield,  emergency  response,  social  network,  etc.)  is  growing 
exponentially.  For  example,  modem  battle  command  is  populated  with  network-centric 
physical  assets;  predominant  among  these  are  constellations  of  MANETs  that  are  used  to 
aid  tactical,  collaborative  communications  during  real-time  battlefield  operations. 
MANETs  represent  a  class  of  battlefield  tactical  communication  networks  that  are  highly 
mobile  and  adaptive  with  respect  to  applications.  MANETs  support  robust  and  efficient 
battlefield  operations,  routing,  communicating,  and  distributing  information 
functionalities  across  their  mobile  nodes. 

Stations  in  MANETs  are  usually  laptops,  Personal  digital  Assistants  (PDAs)  or 
mobile  phones.  These  devices  feature  Bluetooth  and/or  IEEE  802.1 1  (WiFi)  network 
interfaces  and  communicate  in  a  decentralized  manner.  Mobility  is  a  key  feature  of 
MANETs.  Because  of  their  high  cost  and  the  lack  of  flexibility  of  such  networks, 
experimentation  is  mostly  achieved  through  simulation. 

In  the  operational  environment,  MANETs  are  vulnerable  to  enemy  attacks, 
failures  caused  by  engineering  devices,  and  occasional  degradation  due  to  technology. 

All  these  factors  require  that  some  enabling  tools  be  developed  to  support  an  effective 
fielding  of  MANETs  for  command  and  control  (C2)  purposes.  It  also  requires  that 
engineering  analysis  be  conducted  to  monitor  performance  over  time.  Metrics  of 
performance  may  include  vulnerability,  resiliency,  reliability,  trust  between  users,  trust  in 
MANETs,  and  so  on. 

MANETs  constitute  a  special  class  of  networks  that  embrace  humans  and  machines, 
leading  to  what  may  be  described  as  cognitive  socio-technical  systems  (CSTS).  This 
increases  the  complexities  involved  how  interactions  occur  in  systems:  human-human, 
human-machine,  or  machine-machines.  These  qualities  demand  that  MANETs  be 
designed  to  acquire  certain  human  traits  similar  to  how  human  interact  and  behave  in 
dynamic  task  situations.  This  is  the  motivating  factor  for  considering  agent  models  for 
MANETs.  This  consideration  forces  us  to  look  at  important  human  traits  such  as 


perception,  cognition,  behavior,  collaboration,  and  team  work.  Our  agent-based 
simulation  model  incorporates  these  characteristics. 

2.  Intelligent  Agents 

Consider  a  simplified  battlefield  tactical  communication  networks  shown  in 
Figure  1 .  We  may  need  to  know  how  agents  perceive  the  environment  based  on  MANET 
information  load  (voice,  data,  voice  +  data);  how  humans  make  decision  based  on  the 
tactical  requirements  and  supported  by  MANET;  how  multiple  humans  and  multiple 
MANET  users  interact;  and  how  such  interactions  enable  performance.  Answers  to  these 
perceptual,  cognitive,  social,  and  behavior  questions  lead  to  our  interest  in  embodied 
definitions  of  specialized  cognitive  agents  for  human-mobile  network  interaction.  It  is 
possible  to  discover  new  behaviors  as  a  result  of  many  interacting  agent  behaviors. 


Figure  1.  A  simplified  MANET  topology. 

Intelligent  agents  can  then  be  defined  as  a  human  or  a  MANET  device  that  can 
recognize  its  environment  information,  make  sense  of  the  context,  perform  plausible 
reasoning,  and  decide  on  courses  of  action  while  collaborating  with  other  agents.  In  the 
battlefield  environment,  the  agents  should  be  able  to  predict  the  enemy  states  and  plan 
courses  of  action  to  minimize  the  MANET  vulnerability  during  operations.  In  concert 
with  these  assertions,  our  agent  concept  is  to  develop  methods  for  understanding  how 
MANET  operators  exploit  information  in  opportunistic  domains  with  adaptive  and 
dynamic  windows  of  decision  opportunities;  study  how  agent  interactions  between 
MANET  and  humans,  human  and  human,  or  MANET  to  MANET  nodes,  orchestrate 
coping  strategies  during  unexpected  attacks,  uncertainties,  information  sharing,  and  how 
they  collaborate  to  redefine  new  goals  during  system  agitation  states.  Table  1  gives  an 
anectodal  view  of  some  of  the  methods  for  collaboration  by  agents. 


Table  1 

:  Possible  agent  collaboration  modalities 

MANET  device 

Human 

MANET  device 

Instructions  and  rules 

Model-based  predictions 
and  look-up  table 

Human 

User-interface,  visual  tools 

Social-based:  dialogs  and 
communication 

Drawing  from  many  definitions  and  requirement  studies  in  the  agent  community 
(Wooldridge,  1995),  SoS  (Wegner,  et  al.,  2006),  M&S  (Bernstein,  et  al.,  2006) , social 
sciences  and  cognitive  psychology,  an  intelligent  is  likely  to  possess  at  least  one  of  the 
following  properties: 

(1)  Emergence  -the  notion  that  the  interaction  of  technological,  cognitive,  social,  and 
ecological  systems  will  give  rise  to  a  collective  pattern  of  behaviors  that  differs 
remarkably  from  the  presumed  behaviors  from  each  of  sub-systems; 

(2)  Dynamic-  the  notion  that  behavior  change  is  situated  in  time  and  space  given  rise  to 
temporal  and  spatial  behaviors,  respectively; 

(3)  Spiral  model — the  notion  that  due  to  interaction  of  multiple  behaviors,  the  resultant 
system  behaviors  are  non-linear  and  understanding  information  flow  and  their  functions 
is  through  a  continuous  spiral  feedback  model; 

(4)  Self-organized — the  notion  that  agents  that  have  intelligent  can  adapt  and  re-organize 
their  behaviors  for  planning  during  contingencies; 

(5)  Distributed  cognition — the  notion  that  each  agent  in  the  system  share  the  same  goal 
and  seamlessly  distribute  what  they  know  with  each  other; 

(6) .  Sensemaking — the  notion  that  agents  can  reduce  equivocal  information  to  a  common 
metric  for  use  in  an  intended  goal  execution,  and  collectively  seek  prospective 
information  for  coping  with  future  state  changes  (Ntuen,  2006); 

(7) .  Agitative  states — the  notion  that  MANET  agents  in  the  battlefield  operate  under 
stress  levels  which  have  the  effect  of  diminishing  the  full  functioning  of  the  agent’s 
performance  such  as  reduction  of  awareness  and  attention. 

Intelligent  agents  must  be  able  to  learn  (through  various  methods  such  as 
reinforcement,  Bayesian,  feedback,  imitation,  etc).  They  must  exhibit  certain  levels  of 
expertise  based  on  level  of  assigned  experience  on  a  task,  demonstrate  intelligent  or 
ignorance  of  a  subject  matter,  and  be  vulnerable  to  intentional  bias  (Kahneman,  Slovic, 
and  Tversky,  1999).  Premised  on  these  human-like  characteristics,  a  MANET  node  is 
considered  to  represent  an  agent  of  human  and  device  entities.  Thus  a  node  can  exhibit 
behaviors  of  various  forms — from  passive  to  active,  static  to  adaptive,  and  be  capable  of 
demonstrating  selfish-,  collaborative-,  and  participative-,  leadership-,  and  followership- 
behaviors.  A  learning  agent  is  also  constructed  to  have  situation  awareness  capability 
while  interacting  with  its  environment.  Hence,  an  agent  can  monitor  the  behavior  of  other 
agents,  take  commands  from  a  command  and  control  (C2)  agent,  understand  its  roles  and 
when  to  perform  them,  learn  personal  and  organization  level  preferences,  and  be  able  to 
predict  future  actions.  The  objective  of  this  work  is  to  incorporate  these  capabilities  to  the 
MANET  agents 

3.  A  Cognitive  Model  of  MANET  Agents 

The  core  technical  challenge  of  our  work  involves  tackling  cross-disciplinary 
issues  of  dynamic  network  protocol  and  multi-agent  system  design.  An  obvious  approach 
is  to  build  an  agent  out  of  two  (or  more)  subsystems:  a  deliberative  one,  containing  a 
symbolic  world  model,  which  develops  plans  and  makes  decisions  in  a  rational  manner; 
and  a  reactive  one,  which  is  capable  of  reacting  to  events  that  occur  in  the  environment 
without  engaging  in  complex  reasoning.  Often,  the  reactive  component  is  given  some 


kind  of  precedence  over  the  deliberative  one,  so  that  it  can  provide  a  rapid  response  to 
important  environmental  events.  Rather  than  the  classical  approach  of  symbolic 
reasoning,  it  is  assumed  here  that  agent’s  dynamic  behavior  is  a  result  of  interaction  with 
other  agents  and  the  environment  in  which  it  works.  The  agent’s  ability  to  reason  is  not 
necessarily  a  sufficient  condition  for  sensemaking,  but  the  resultant  behaviors  arising 
from  interactions — socially  and  ecologically. 

One  novel  approach  to  representing  intelligent  behaviors  to  MANET  agents  is  to 
use  the  OODA(Observe,  Orient,  Decide,  Act)  model.  This  is  shown  in  Figure  2  for  a 
four-node  MANET  system.  The  OODA  model  was  developed  by  Boyd  (1987)  to  address 
the  concerns  of  military  decision-making  processes  that  consider  uncertainties.  In  the 
OODA  model,  the  “Orient”  sub-model  attempts  to  capture  the  cognitive  processes 
involved  during  sensemaking — although  it  was  never  addressed  as  such.  The  components 
describe  the  human  cognitive  tasks  with  feedback  and  feed-forward  loops.  Boyd 
describes  the  sensemaking  process  in  four  stages  with  the  orientation  stage  being  the 
stage  at  which  most  of  the  sensemaking  process  takes  place. 
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Figure  2.  Multiple  connected  OODA  for  MANET  agents 

The  multiple  connected  OODA  network  is  modeled  as  a  propositional  network. 
According  to  Black  [5],  propositional  networks  are  sets  of  entities  illustrated  by  their 
interrelated  relations  and  properties.  It  is  similar  to  concept  map  in  that  they  both 
represent  knowledge  structures  and  contents  in  a  declarative  way.  The  causal  relations  in 
concept  map  can  also  be  represented  as  a  series  of  condition-action  or  if-then  rules. 


The  model  framework  is  based  on  shared  ontology.  Ontology  is  often  defined  as 
an  “explicit  specification  of  a  conceptualization”  (Fox  &  Gruninger,  1997;  Gruber,  1993). 
It  is  used  to  define  and  organize  knowledge  concepts  in  a  system.  Thus,  ontology  can  be 
regarded  as  a  tool  for  information  network  management.  The  ontology  approach  for  the 
multiple  OODA  network  is  simple:  When  we  perform  actions,  we  do  so  after  a  careful 
selection,  often  through  rational  model,  of  interrelated  activities. 

The  spatial  connection  or  relationship  of  nodes  (with  each  node  with  agent 
properties)  and  their  intentional  forms  a  network,  referred  to  here  as  a  Knowledge  Action 
Network  for  Agents  (KANA).  KANA  is  a  construct  similar  to  Black’s  propositional 
network,  except  that  KANA  allows  for  representation  of  retrospective,  current,  and 
prospective  sensemaking  schemata  for  agents  so  as  to  enable  reflexive  and  envisioning 
reasoning  behaviors.  KANA  also  allows  representation  for  agents  to  collaborate  during  a 
reasoning  process  to  manage  uncertain  system  resources  to  achieve  intended  system-level 
actions.  KANA  is  designed  to  capture  and  build  behavior  relations  between  agents 
employing  well  matured  cognitive  tools  such  as  activity  theory,  cognitive  task  analysis, 
behavior  analysis,  functional  requirement  specification,  and  operation  tracing  and 
mapping.  For  example,  social  behavior  ontology  should  be  able  to  represent:  how 
interactions  occur  among  agents;  how  collaborations  take  place;  how  agents  negotiate 
and  reach  consensus;  and  how  leaders  are  selected  among  agents.  A  more  global  ontology 
may  represent  how  variability  in  agent  behaviors  are  captured  over  time  (temporal),  space 
(spatial),  location  of  activity  (situational),  and  effect  of  workload  (conditional).  KANA 
also  addresses  human  dimensions,  conditions  and  rules  for  behavior  representations, 
autonomy,  and  intention  (shared  and  individual),  and  characteristics  of  SoS  at  different 
platforms  and  design  hierarchies  (e.g.,  soldier,  tank,  platoon,  etc.)  as  explicated  on  the 
battlefield  (Cioppa,  Lucas,  &  Sanchez,  2004). 

We  consider  MANET  agents  to  learn  and  understand  the  human  user  actions. 
These  agents  should  be  able  to  Predict,  Envision,  Anticipate,  Reason,  and  Learn 
(PEARL).  This  leads  to  the  concept  of  PEARL  as  a  high-level  meta-agent  for 
supervising  the  behaviors  of  other  agents.  They  can  use  what  they  have  learned  from 
interactions  to  determine  what  to  do  in  the  future  within  its  environment.  The  learning 
agent  watches  out  for  themselves,  enforcing  their  own  individual  preferences  and  taking 
advantages  of  others  preferences  and  biases.  PEARL  has  a  three-stage  interacting  layer:  a 
currency  layer  which  has  some  or  all  information  about  the  current  state  of  the  system;  a 
retrospective  sensemaking  layer  which  has  some  or  all  the  pertinent  information  about  the 
past  system  behaviors  and  performance  measures,  including,  e.g,  beliefs  either  about  the 
external  world  or  the  system’s  internal  state;  and  prospective  (envisioning)  state  which 
has  models  of  a  system  and  its  agent  behaviors.  The  success  of  PEARL  lies  on  how 
information  is  shared  between  agents.  Thus,  PEARL  allows  agents  the  ability  for 
perceptual-control  of  dynamic  actions  through  learning,  shifting  system  goals,  time 
pressure,  information  dynamics,  complex  battlefield  operations  requiring  adaptive 
courses  of  actions,  skill  acquisition,  adaptive  and  self-organizing  behaviors,  visualizing 
problem  space,  and  hierarchical  multilayer  interaction  of  command  and  control  tasks 
between  and  among  agents  (human  and  software)  performing  tasks,  sharing  resources, 
and  self  organizing— including  exchanging  leadership  roles  in  the  cell — such  as  switching 
roles  and  building  redundancy  systems  (self-satisficing  solutions)  during  high  workload 


in  the  SoS;  and  autopoiesis— a  self-production  -  maintenance  of  a  living  organism's  form 
with  time  and  flows. 

4.  SAMPLE  SIMULATION 


We  are  developing  two  agent-based  environments.  The  first  is  a  constructive 
simulation  based  on  network  and  graph  theory.  Here  the  user  can  create  arbitrary  number 
of  MANET  nodes  and  conduct  the  simulation  experiment  over  a  stipulated  use  time.  This 
is  shown  in  Figure  3a.  For  each  node,  the  user  assigns  the  agent  roles  such  as  scouting, 
security  operation,  etc.,  node  potential  failure  mechanisms,  a  list  of  potential  vulnerable 
factors  (e.g.,  jamming,  disruption,  spoofing,  attack,  fake  routing  message,  interceptions, 
etc.)  and  the  risk  factors  for  each  events.  The  collection  of  agent  properties  is  shown  in 
Figure  3b.  The  simulation  can  produce  statistics  as  queried  by  the  user.  For  example,  in 
Figure  4,  statistics  on  C2  manager  (from  PEARL  agent)  is  produced.  In  Figure  4  PEARL 
agent  can  produce  situation  watch  statistics  such  the  net  intrusion  (89.3%),  device  failures 
(33.8%),  communication  failure  (85.5%),  and  discrepancy  in  messages  (38.8%).  PEARL 
also  identifies  enemy  spying  into  the  network  (91%),  listening  to  communication  (20%), 
and  attempted  attacks  on  the  network  (61%).  PEARL  agent  uses  cause-effect  and  pattern 
mapping  to  predict  the  network  behaviors.  For  instance,  the  simulation  showed  that  loss 
of  information  was  detected  30.3%,  information  degraded  about  73.7%,  and  there  were 
critical  changes  in  node  behaviors  (83.7%).  The  consequences  for  these  actions  are 
identified.  For  instance,  loss  of  safety  occurred  73%  of  the  time  these  events  happen.  The 
vulnerability  of  the  network  in  this  example  is  66.8%— meaning  that  the  network  is 
vulnerable  to  outside  incursions  or  likely  to  fail  in  performing  its  duty  66.8  percent  of  the 
time. 
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Figure  3a.  Sample  MANET  nodes 
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Figure  3b.  A  window  to  capture  agent  properties 


Figure  4.  Sample  output  statistics  from  PEARL  agent 

4.  Performance  Evaluation  of  a  Battlefield  Visualization  Tool 

The  second  part  of  the  project  is  the  experimental  domain  which  seeks  to  mimic 
MANET  behaviors  using  a  collaborative  sensemaking  software  system  (S3)  developed  by 
Ntuen,  Park,  and  Kim  (2008).  Here,  we  allow  up  to  three  users  and  computers  that  are 
remotely  (geographically  dispersed)  within  our  laboratory  to  serve  as  different  MANET 


nodes.  The  purpose  of  this  is  to  enable  real-time  technology  transfer.  At  the  present  time, 
Figure  5  shows  and  example  experimental  simulation  for  direct  and  active  MANET 
configuration.  In  Figure  5,  two  MANET  nodes  (users)  and  C2  (PEARL)  are  configured 
as  shown  on  the  upper  part  of  the  Figure  5.  The  lower  part  of  Figure  5  shows  sample 
window  that  captures  agent  (Node)  information  in  real-time  information.  Figure  6  shows 
an  example  of  how  PEAR  recognizes  intruder  by  asking  for  node  entry  verification. 
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Figure  5.  Sample  MANET  agent  experimental  frame. 


Figure  6.  Network  intrusion  detection  and  verification  by  PEARL  agent 

4.  Summary  and  Conclusion 

MANETs  constitute  a  special  class  of  networks  that  embrace  humans  and 
machines,  leading  to  what  may  be  described  as  cognitive  socio-technical  systems  (CSTS). 
This  increases  the  complexities  involved  how  interactions  occur  in  systems:  human- 
human,  human-machine,  or  machine-machines.  Modeling  MANET  from  the  standpoint 
of  system  of  systems  is  the  focus  of  our  on-going  project.  The  current  results  are 
anecdotal  with  respect  to  scale-down  properties  scoped  for  demonstration  of  the  efficacy 
of  agents  in  a  system  of  systems  (SoS)  MANETs.  From  our  pilot  study,  two  fundamental 
meta  problems  are  constraints  to  realistic  cognitive  modeling  and  representation  of  agent 
properties.  The  first  is  dealing  with  adaptive  behaviors  as  a  consequent  of  information 
changes  from  battlefield  tasks  and  the  supporting  mobile  wireless  communication 
networks.  KANA  knowledge  manager  in  our  system  is  designed  to  manage  this  kind  of 
situation  in  large-scale  complex  networks.  The  second  challenge  is  reducing  complicated 
and  complex  human  observable  behaviors  to  simple  qualitative  rules  for  agents  to  learn. 
We  achieved  this  by  using  decoupled  OODA  models. 

In  our  simplified  experiments  with  PEARL  agent,  we  can  compute  useful 
MANET  network  properties  such  as  vulnerability,  resiliency,  and  reliability.  We  are  not 
dealing  with  the  typical  network  statistical  characteristics  such  as  centrality.  Hence,  there 
are  other  human-centric  agent  properties  to  be  added.  These  include,  but  are  not  limited 
to,  how  agents  believe  each  other  as  a  function  of  stereotypical  and  bias  knowledge, 
social  affinity  as  a  function  of  team  situation  awareness  and  collaboration,  how  agents 
bind  problems  in  context  and  provide  solutions  when  faced  with  uncertainties  and 
surprise,  how  agents  learn  (e.g.,  what  important  factors  make  an  agent  to  use,  say, 
reinforcement  learning  as  opposed  to  imitation  learning),  the  kinds  of  emerging  behaviors 
when  an  agent  interacts  with  other  agents  in  context,  and  the  level  and  probability  of  self- 


organization  by  agents  when  a  network  is  attacked.  These  issues  are  critical  to  the 
survivability  of  MANETS  in  battlefields. 

Our  simplified  simulation  further  inform  that:  (a)  if  decision  making  in  a 
dynamic  battlefield  problem  solving  environment  is  to  be  driven  by  simulation,  it  is 
necessary  to  develop  real-time  models  than  can  self  organize  in  response  to  new 
information;  (b)  SoS  simulation  inherits  special  properties  of  advanced  distributed 
simulation  which  requires  rapid  information  processing  and  manipulation  of  extremely 
large  information;  and  (c)  when  multiple  entity  behaviors  interact,  it  is  possible  to  derive 
latent  intelligent  behaviors  that  make  the  functioning  of  SoS  scalable  across  different 
echelons  of  information  abstraction  and  control.  These  are  the  basic  research  problems 
our  agent-based  environment  will  address  as  an  on-going  research. 
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BACKGROUND 


MANET:  A  popular  acronym  for  Mobile  Ad  hoc 

S A  MANET  is  an  autonomous  collection  of 
mobile  users  that  communicate  over 
relatively  bandwidth  constrained  wireless 
links. 

S Since  the  nodes  are  mobile,  the  network 
topology  may  change  rapidly  and 
unpredictably  over  time. 

S  The  network  is  decentralized,  where  all 
network  activity  including  discovering  the 
topology  and  delivering  messages  must  be 
executed  by  the  nodes  themselves,  i.e., 
routing  functionality  will  be  incorporated 
into  mobile  nodes. 

S A  hybrid  of  human-machine-  or  machine- 
machine-  system 


NETwork 


www.&hutterstoch  cam  -  42330161 


BACKGROUND 

Mobile 

-  Random  and  perhaps  constantly 
changing 

Ad-hoc 

-  Not  engineered 

Networks 

-  Elastic  data  applications  which 
use  networks  to  communicate 

Ad  hoc  networks: 

Do  not  need  backbone  infrastructure 
support 

Are  easy  to  deploy 

Useful  when  infrastructure  is  absent, 

destroyed  or  impractical 


•  Interconnected  collection 
of  wireless  nodes 

•  Nodes  enter  and  leave 
over  time 

•  Nodes  also  act  as 
routers;  forward  packets 

•  No  pre-established 
network  infrastructure 

•  No  centralized 
administration 

•  Communication  using 
BlueTooth  and  WAP 


Technology  Development 


Envisioned  Evolution  of 


Self-configuring 

Self-organizing 

Self-healing 

Self-managing 

Increasing  Capability 

From  :  Brian  Adamson,  NRL 


Many  Applications  of  MANET 

Personal  area  networking 

-  cell  phone,  laptop,  ear  phone,  wrist  watch 
Military  environments 

-  soldiers,  tanks,  planes 
Civilian  environments 

-  taxi  cab  network 

-  meeting  rooms 

-  sports  stadiums 

-  boats,  small  aircraft 
Emergency  operations 

-  search-and-rescue 

-  policing  and  fire  fighting 


Military  applications 


Combat  regiment  in  the  field 

-  Perhaps  4000-8000  objects  in 
constant  unpredictable  motion... 

Intercommunication  of  forces 

-  Proximity,  function,  plan  of  battle 
Special  issues 

-  Low  probability  of  detection 

-  Random  association  and  topology 


4 - Domain 


Domain 


Domain 


Challenges  in  Mobile 
Environments 

Limitations  of  the  Wireless  Network 

•  packet  loss  due  to  transmission  errors 

•  variable  capacity  links 

•  frequent  disconnections/partitions 

•  limited  communication  bandwidth 

•  Broadcast  nature  of  the  communications 

Limitations  Imposed  by  Mobility 

•  dynamically  changing  topologies/routes 

•  lack  of  mobility  awareness  by  system/applications 

Limitations  of  the  Mobile  Computer 

•  short  battery  lifetime 

•  limited  capacities 


Challenges  Continue 

► 

•  Dynamic  Topologies  and  node  memberships 

•  Bandwidth  constraints 

•  Many  Transmission  Errors 

•  Energy-constrained  operation 
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Community  Attention  to  Manets 

•  Routing/  packet  scheduling 

•  Reliability 

•  Lethality 

•  Energy  consumption  and  longevity 

•  Vulnerability 

•  Mobility 

•  Security 

•  Survivability 


Motivation 

MANET  as  a  human-machine  system 


•  MANETOLOGY:  Develop  a  network  theory 
for  human-machine  system  (with  MANET  = 
machine) 

1 .  Allows  for  modeling  of  fundamental  human 
characteristics  in  intelligent  agent-based  networks. 

2.  Allows  for  representation  framework  for  CSTS 

3.  Advance  cognitive  network  theory  for  modeling  and 
simulation 

Question:  Does  agent-based  MANET  performance 

(measured  by  vulnerability)  affected  by  human  traits  like 
behavior,  perception,  and  cognition  abilities? 


INFLUENCING  FACTORS  FOR 

MANETOLOGY 


(1)  Emergence  -  the  notion  that  the  interaction  of  a  technological,  cognitive,  social, 
and  ecological  system  will  give  rise  to  a  collective  pattern  of  behaviors  that  differ 
remarkably  from  the  presumed  behaviors  from  each  of  the  sub-systems; 

(2)  Dynamic  -  the  notion  that  behavior  change  is  situated  in  time  and  space  giving 
rise  to  temporal  and  spatial  behaviors,  respectively; 

(3)  Spiral  model  -  the  notion  that  due  to  the  interaction  of  multiple  behaviors, 
resultant  system  behaviors  are  non-linear,  and  understanding  information  flow  and 
their  functions  are  mediated  through  a  continuous  spiral  feedback  model; 

(4)  Self-organized  -  the  notion  that  agents  that  have  intelligence  can  adapt  and  re¬ 
organize  their  behaviors  for  planning  during  contingencies; 

(5)  Distributed  cognition  -  the  notion  that  each  agent  in  the  system  share,  the  same 
goal  and  seamlessly  distribute  what  they  know  with  each  other; 

(6)  Sensemaking  -  the  notion  that  agents  can  reduce  equivocal  information  to  a 
common  metric  for  use  in  an  intended  goal  execution,  and  collectively  seek 
prospective  information  for  coping  with  future  state  changes  (Huang  &  Chang,  2006); 

(7)  Agitative  states  -  the  notion  that  agents  for  military  M&S  will  operate  under  stress 
levels  which  have  the  effect  of  diminishing  the  full  functioning  of  the  agent’s 
performance  such  as  reduction  of  awareness  and  attention. 


APPROACH— MANET  AS  A  COGNITIVE 
SOCIO-TECHNOLOGY  SYSTEM  (CSTS) 
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At  each  node,  the  human  activities  are  to 
Observe,  Orient,  Decide,  Act 


MANET  device 
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Automated  behaviors 

Model-based  predictions 
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User-interface,  visual 
tools 


Social-based:  dialogs  and 
communication 
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The  OODA  model  was  developed  by  Boyd  (1987) 


APPROACH— MANET  AS  A  COGNITIVE 
SOCIO-TECHNOLOGY  SYSTEM  (CSTS): 

Why  Agents 

(a)  cope  with  complex  interaction  of  multiple  behaviors; 

(b)  capable  of  analyzing  complex  adaptive  information; 

(c)  cope  with  contingencies  under  emergence  behaviors  and 
events; 

(d)  recognize  opportunities  in  a  spatio-temporal  manner; 

(e)  seek  satisficing  and  plausible  (good  enough)  solutions 
when  confronted  with  unexpected  situations  with  uncertain 
and  equivocal  information; 

(f)  represent  as  much  as  is  feasible  the  various  dimensions  of 
expert  knowledge  in  the  domain  problems 


APPROACH:  Agents  in  MAN  El 


Assume  the  basic  principle  of  a  Rational  Agent:  For  each 
possible  percept  sequence,  a  rational  agent  should  select  an 
action  that  is  expected  to  maximize  its  performance  measure 
given  the  evidence  provided  by  the  percept  sequence  and 
whatever  built-in  knowledge  the  agent  has. 
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APPROACH:  Agents  in  MAN  El 


Russell  &  Norvig  (2003).  Artificial  Intelligence:  A  Modern  Approach;  Prentice  Hall. 

Agents  can  perform  actions  in  order  to  modify  future 
percepts  so  as  to  obtain  useful  information 
(information  gathering,  exploration). 


APPROACH:  Modeling  Representation 
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Environment 


Agent 

•  Attributes 

•  Behavioral  rules 

•  Memory 

•  Resources 

•  Decision  making  sophistication 

•  Rules  to  modlify  behavioral  rules 
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Each  agent  interacts  (directly  or  indirectly)  with  one  or 
more  aspects  of  an  environment. 


APPROACH:  Modeling  Representation 


Proc.  of  8th  Conference  on  Computer  Generated  Forces  and  Behavioral  Representation,  Orlando,  FL,  May  1999 


Modeling  Perceptual  Attention  in  Virtual  Humans 

Randall  W.  Hill,  Jr. 

Agent  Environments 

Fully  vs.  Partially  Observable  (Accessible  vs.  inaccessible) 
Deterministic  vs.  Stochastic  (non-deterministic) 

Episodic  vs.  Sequential  (non-episodic) 

Static  vs.  dynamic 
Discrete  vs.  continuous 


APPROACH:  Modeling  Representation 
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APPROACH:  How  We  Do  It 


event 

evaluation 


axpectad 

awanl 


overt 


gonli-snd 

ddbrDlj 


«Mp«ClDQCn 
dnconlrmnliai! 

.r- 


aV*Ti1 
■mpcrianc# 


want 

njHtJly 


s  orprise 
generation 


wai  ±il  irvjiortinca 

.novafcj 


furpnu 

inlAn-bity 


0 

■d*:iy 


process 

:  an:  ^Tiil-iriG  :Url  thraihold 

,G4n»making  and  Ihrashokl 


effect  Of 
senserr,aking 


yheadtoT"'" 
/£aiK*  making 

durdticri  dC 

-Cy aamamahing 

SAncArmking 

T\faadbaclc 

baliakon 

siluilion 


marnory 


aan  landing 
abiity 


lima 
premium 
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Agent  Behaviour  in  Simulated  Environments 
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APPROACH:  Considering  Behavior 
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APPROACH:  Modeling  Behavior 


belief 

knowledge 


pro-attitudes  { 


desire 

intention 

obligation 

commitment 

choice 


Intelligent 


and 


Michael  Wooldridge 


Nicholas  R.  J  c  11  n  i  n 


An  agent  is  completely  specified  by  the  agent 
function  mapping  percept  sequences  to 
actions.  We  use  a  model-based  reflex  agent 

function  paradigm  for  the  prototype 
simulation. 


PEARL  SIMULATION  ARCHITECTURE 


Scenarios. 


Predict,  Envision,  Anticipate,  Reason,  and 
Learn  (PEARL) 


KANAN: 

Knowledge  action 
repository 


SOAP  (Simulation  Of  Agent  Perception); 

SOAC  (Simulation  Of  Agent  Cognition); 

SOAS  (Simulation  Of  Agent  Socio-culture);  and 
SOAB  (Simulation  Of  Agent  Behaviors). 


SOAP: 

Perception  of 
Environment 
(Level  1  SA) 


SOAC: 

Agent  cognition 
of  the  problem  space 


SOAB: 

Simulation  model  for 
Agent  behaviors 


SOAS: 

Social  and  cultural 
cognition 


An  agent  function  can  have  one  or  all  of: 

Simple  reflex  agents:  If  the  world  is  X  then  action  Y 

Goal-based  agents:  For  situation  X  what  should  I  do  to  achieve  Y? 
Utility-based  agents:  If  I  do  X  for  situation  Y,  my  satisfaction  is  Z  >  Q 


SIMULATION  (Has  a  Suite  of  36  Major 

Algorithms) 

SAMPLE  Behavior  Adaptation  Algorithms 

1.  Agent  ID 

2.  Time  :  The  time  agent’s  properties  reported  to  the  command  node. 

3.  Roles  :  Agent’s  role  assigned  by  Command  Node. 

4.  Physical  Location  (X,Y,Z) :  Agent’s  Current  Location  on  the  Real  Map(Google  Map).  (Z=  Zoom  level) 

5.  Behavior_F  :  get  from  ‘probability  of  failure’  received  from  agent  node  (  min  +  (max  -  min)*rand() ) . 

6.  Behavior_A  :  get  from  ‘probability  of  attack’  received  from  agent  node  (  min  +  (max  -  min)*rand() ) . 

7.  Behavior_AD  :  Adaptability  when  there  is  enemy  attack. 

(  yadap  =  (2  /  (1+e'kf<h’°>  )  )  -  1 

k  =  1,  f(h,c)  =  Trapezoidal  Fuzzy  Number  using  hostility(h)  and  capability(c)  level 

if  yadap  <  0  then  :  Agent  is  Not  Adaptive 

if  0  <=  yadap  <  0.4  then  :  Agent  is  Sluggishly  Adaptive 

if  0.4  <  yadap  <=1.0  then  :  Agent  is  Adaptive 

8.  Perception  :  get  from  ‘Situation  Awareness  ability’  received  from  agent  node  (  min  +  (max  -  min)*rand() ) . 

if  0.5  <  SA  <=  1 .0  then  :  Recognize 
if  0.0  <=  SA  <  0.5  then  :  Fail 


9.  Learning  :  (reinforcement  learning,  discounted  time  learning) 


SAMPLE  SIMULATION  RESULTS 
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■S  PEARL  (Predict,  Envision,  Anticipate,  Reason,  and  Learn) 


Arc  size  defines 

frequency  of  node-  to-  node  interaction 


Environment 
|  Desert 


Difficulty  Index:  |q9 
Agitation  (P) 


Probable  Value  of  Node  G  m.78 


Battlefield  Operations 


Task  :  jpiatoon  Network  ^~| 
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Add  Done  I 


Events 

Event:  I  [Select] 
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Nodes  Relationship : 
Nodes(1,2]  =  0.062 


Input 

parameters 


ulnerability 
of  network 
during 
simulation 
=  69% 


Node(1)  Eventfl] : 

-  Platoon  Network 
Node(2)  Eventfl] : 

-  Platoon  Network 


Sample  network  topology 
(A  MANET  with  6  nodes; 
allowed  number  of  nodes 
is  arbitrary) 


Sample  node  intensity  (45.6%) 
calculated  as  aggregated 
parameter  effects:  task  difficulty, 
interaction  requirements,  perception  of  environment, 
personality  type,  etc. 


SIMULATION  RESULTS 


SO  AC  Stat. 


Enemy  Activity 

1 .  Intruding 

2.  Spying 

3.  Listening  to  Communication 

4.  Attacking  (Network) 

5.  Mimicking 


System  Status  at  Time  —  5 

C2  Activity 
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1 .  Information  Flow 

2.  Network  Behavior 

3.  Intruder 

4.  Discrepancy 

5.  Device  Failure 

6.  Communication  Failure 
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5.  Disruption  of  Services 

13 

F 

F 

F 

F 

F 

F 

6.  Loss  of  Equipment 
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Behavior 

Agent  1 


A.  Prob  (Agent  is  Proactive  and  Active) 

B.  Prob  (Agent  is  Proactive  and  Passive) 

C.  Prob  (Agent  is  Reactive  and  Active) 

D.  Prob  (Agent  is  Reactive  and  Passive) 


A  B  C  D 


t=0 

0.20 

0.04 

0.65 

0.12 

t=1 

0.17 

0.01 

0.78 

0.05 

t=2 

0.00 

0.00 

0.95 

0.05 

t=3 

0.24 

0.06 

0.57 

0.14 

t=4 

0.01 

0.00 

0.91 

0.08 

t=5 

0.03 

0.00 

0.88 

0.09 

Behavior  Map  Predictions  (‘Vo) 


E. 

Simulated  likelihood  Active  behavior 
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Simulated  likelihood  Passive  behavior 
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Simulated  likelihood  Proactive  behavior 
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H. 

Simulated  likelihood  Reactive  behavior 

t=5 

59.34 
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SIMULATION  RESULTS:  Agent  Learning 

Profiles 


Forgetting  is  triggered  by  task  conditions  that  disable  rational  and  deliberate  mental 
models  -forcing  the  agent  to  ignore  (or  forget)  routine  processes. 


Positive  reinforcement  is  earned  by  an  incremental  credit  awarded  to  an  agent 
for  routinely  achieving  an  intended  goal. 


APPLICATION  OF  SIMULATION  RESULTS 
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APPLICATION  OF  SIMULATION  RESULTS 


Human  injury 
reported  by  agent 
at  MANET  node  2 


Injury  report  verification  by  C2  server  to  avoid  enemy 
mimicking  node  2  behavior  or  status 


APPLICATION  OF  SIMULATION  RESULTS 


Normalized  %  Z  Score 


SIMULATION  RESULTS  (Agent  1) 


Vulnerability 

■  Perception 
Behavior 

■  Cognition 


Simulation  Run  (480  mins  each) 


Vul 

Percep 

Behavior 

Cognition 

Vul 

-0.991 

-0.198 

-0.512 

Percep 

0.319 

0.509 

Behavior 

0.54 

Cognition 

Pearson 
Correlation  for 
Simulated  Period 


SIMULATION  RESULTS  (Agent  1) 


Radar  Plot  of  Average 
Normalized  % 

Scores  (low  =  0.0,  high  =  1 .0) 

^ Agent  cognition  more 
influential. 

s Cognition  correlates  positively 
with  perception  and  behavior. 
^Decreased  vulnerability  = 
increased  scores  in  cognition, 
behavior,  and  perception 


i 


Vulnerability 

■Perception 

Behavior 

■Cognition 


Vulnerability 

Perception 

Behavior 

Cognition 

0.25 

0.7 

0.56 

0.8 

0.1 

0.9 

0.77 

0.85 

0.15 

0.88 

0.8 

0.75 

0.56 

0.4 

0.7 

0.66 

0.35 

0.65 

0.8 

0.9 

SUMMARY  AND  CONCLUSION 


1 .  Modeling  MANET  as  a  cognitive  socio-technical 
system. 

2.  MANET  players  considered  collaborative  agents: 

3.  Applied  network  science  to  capture  MANET  nodes 
as  cognitive  agents 

4.  Inject  human  cognitive  and  behavioral  traits  into 
agent-based  modeling  and  simulation 

5.  Use  OODA  model  and  sensemaking  paradigms  to 
drive  non-deliberate  behavior  of  agents  as  rational 
entities  (model-based  functions). 

6.  Experiment  with  positive  reinforcement  learning 
(with  incremental  gain  over  time),  and  learning  with 
forgetting  caused  by  task  changes). 


SUMMARY  AND  CONCLUSION 

7.  Baseline  Research  Question:  Does  an  agent-based 
MANET  performance  (measured  by  vulnerability) 
affected  by  human  traits  like  behavior,  perception, 
and  cognitive  abilities? 

(a)  As  agents  gain  and  exhibit  increasing  perception 
of  the  problem  situation,  show  positive  rational 
behaviors,  and  gain  expertise  (cognition),  MANET 
nodes  are  less  likely  to  show  high  vulnerability 
during  a  mission. 

(b)  Agents  exhibit  cognition,  perception  and 
behavior  traits  that  are  positively  correlated. 

(c)  Agents  exhibit  more  human  cognitive  traits  in 
solving  problems  (learning  and  forgetting  co-exist). 


SUMMARY  AND  CONCLUSION 

8.  Have  demonstrated  the  utility  of  the  model  for  use 

in  training: 

S  MANET  node  performance  statistics. 

S  Human  performance  as  orchestrated  by  system 
interactions. 

S  Levels  of  collaboration/  information  sharing 
during  system  level  mission. 

9.  Embellish  PEARL  model  with  other  agent 
functional  algorithms;  extend  to  system-of 
systems  modeling;  compare  performance. 

10.  Conduct  field  test  to  measure  effects  on 
survivability,  vulnerability,  lethality,  and  system 
reliability. 


